Knowledge Distillation (KD) is a powerful approach for compressing a large model into a smaller, more efficient model, particularly beneficial for latency-sensitive applications like recommender systems. However, current KD research predominantly focuses on Computer Vision (CV) and NLP tasks, overlooking unique data characteristics and challenges inherent to recommender systems. This paper addresses these overlooked challenges, specifically: (1) mitigating data distribution shifts between teacher and student models, (2) efficiently identifying optimal teacher configurations within time and budgetary constraints, and (3) enabling computationally efficient and rapid sharing of teacher labels to support multiple students. We present a robust KD system developed and rigorously evaluated on multiple large-scale personalized video recommendation systems within Google. Our live experiment results demonstrate significant improvements in student model performance while ensuring consistent and reliable generation of high quality teacher labels from a continuous data stream of data.

On the heels of a mostly positive jobs report from the Bureau of Labor Statistics (BLS) (4.6% unemployment is the lowest it's been in nine years), the McKinsey Global Institute (MGI) released a more sobering snapshot of the world of work. A briefing by MGI director James Manyika, compiled from the company's extensive research, took a deeper dive into employment numbers. In the United States and the 15 core European Union countries (E.U.-15), there are 285 million adults who are not in the labor force--and at least 100 million of them would like to work more. Some 30% to 45% of the working-age population around the world is underutilized--that is, unemployed, inactive, or underemployed. Manyika says that unemployment figures typically get the most attention at the expense of those who are underemployed.